Adaptive Camera Control and Calibration For Dynamic Focus

ABSTRACT

A camera vision system for creating 3D reconstructions of objects may include a camera, a distance sensor having a fixed spatial relationship with the camera, and a system controller. The system controller receives distance sensor signals from the distance sensor indicating a sensor-to-object distance, determines a camera-to-object distance and a corresponding camera focus state based on the sensor-to-object distance, and transmits camera focus state control signals to cause the camera to adjust to the camera focus state. The system controller retrieves camera intrinsic parameter values for the camera focus state, transmits image capture control signals to cause the camera to capture an object image of the object, receives object image data from the camera for the captured object image, and stores the object image data and the camera intrinsic parameter values in an image database for use in the 3D reconstruction.

TECHNICAL FIELD

The present disclosure relates generally to camera visions system and,more particularly, to dynamic camera focusing and retrieval of intrinsiccamera calibration parameters based on a sensed distance from the camerato an object.

BACKGROUND

Camera vision systems, such as stereo vision systems and structuredlight, are used throughout various industries to reconstructthree-dimensional (3D) scans of objects for various purposes. For highresolution 3D reconstruction, intrinsic camera calibration parameters,including focal length and distortion coefficients, must be known atpixel-level accuracy for a given focus state of a camera in order togenerate an accurate and detailed reconstructed image of an object.Changes in the focus state of the camera (manual focus changes ordynamic focusing) are often required to take quality high resolutionimagery in situations in which the distance from the camera to theobject varies. The intrinsic parameters to be used for reconstructingcaptured images of the object into the 3D image are dependent on thefocus state of the camera, so the correct intrinsic parameters must beselected when the focus state changes.

SUMMARY OF THE DISCLOSURE

In one aspect of the present disclosure, a camera vision system forcreating a 3D reconstruction of an object. The camera vision system mayinclude a camera having a camera controller, a distance sensor having afixed spatial relationship with the camera, and a system controlleroperatively connected to the camera controller and the distance sensor.The system controller may be programmed to receive distance sensorsignals from the distance sensor, wherein the distance sensor signalsindicate a sensor-to-object distance from the distance sensor to theobject, determine a camera-to-object distance from the camera to theobject and a corresponding camera focus state for the camera-to-objectdistance based on the sensor-to-object distance, and transmit camerafocus state control signals to the camera controller to cause the cameracontroller to adjust the camera to the camera focus state. The systemcontroller may further be programmed to retrieve camera intrinsicparameter values for the camera that correspond to the camera focusstate, transmit image capture control signals to the camera controllerto cause the camera to capture an object image of the object, receiveobject image data from the camera controller corresponding to the objectimage captured by the camera, and store the object image data and thecamera intrinsic parameter values in an image database.

In another aspect of the present disclosure, a method of dynamicallyfocusing a camera and capturing images and generating 3D reconstructionsof objects. The method may include determining a first camera-to-objectdistance from the camera to an object, causing a camera controller ofthe camera to adjust a camera focus state of the camera to a firstcamera focus state that corresponds to the first camera-to-objectdistance, retrieving first camera intrinsic parameter values of thecamera that correspond to the first camera focus state, commanding thecamera controller to cause the camera to capture a first image of theobject, and storing first image data of the first image and the firstcamera intrinsic parameter values.

In a further aspect of the present disclosure, a method for performingintrinsic calibration in a camera vision system. The method may includeadjusting a camera of the camera vision system to a camera focus state,capturing an image of a target object, determining intrinsic parametervalues for the camera at the camera focus state from captured targetimage data for the image of the target object, and storing the camerafocus state and the intrinsic parameter values for the camera at thecamera focus state in an intrinsic parameter database.

Additional aspects are defined by the claims of this patent.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic illustration of components of a camera visionsystem in accordance with the present disclosure;

FIG. 2 is a schematic illustration of the camera vision system of FIG. 1with a camera and a distance sensor moved to a second position relativeto an object;

FIG. 3 is a schematic illustration of the camera vision system of FIG. 1with a second camera; and

FIG. 4 is a block diagram of control components that may implement imagecapture and 3D reconstruction in accordance with the present disclosurein the camera vision system of FIG. 1;

FIG. 5 is a flow diagram of an exemplary camera vision system intrinsiccalibration routine in accordance with the present disclosure; and

FIG. 6 is a flow diagram of an exemplary image capture and 3Dreconstruction routine in accordance with the present disclosure.

DETAILED DESCRIPTION

Camera vision systems in accordance with the present disclosure providean adaptive intrinsic calibration method to support dynamic focus of thecamera system for acquisition of image data to be used in 3Dreconstruction of an imaged object. In an exemplary embodiment, thecamera vision system includes one or more cameras and a correspondingdepth or distance sensor mounted on a mobile structure such as apan-tilt unit. As the cameras pan and/or tilt, the distance from thecamera to the target object change so that the depth of field is nolonger centered about the target object without updating the focus ofthe camera. The distance sensor senses the distance to the object, andthe system determines a new focus state for the camera corresponding tothe sensed distance and adjusts the focus of the cameras accordingly. Asa focus changes, intrinsic parameters of the cameras for the new focusstates are retrieved for use in converting captured image data to a 3Drepresentation of the object. This arrangement can improve the speed andquality of the 3D reconstruction of the object over previous systems.

Referring to FIG. 1, an exemplary camera vision system 10 is illustratedthat is capable of capturing images of an object 12 and creating a 3Dreconstruction of the object 12 from the captured images. The cameravision system 10 may include a camera 14, a distance sensor 16 and asystem controller 18 operatively connected to the camera 14 and to thedistance sensor 16 for exchanging information that will control theoperation of the camera 14 to capture images of the object 12. In thecamera vision system 10, the camera 14 and the distance sensor 16 have afixed spatial relationship within a three-dimensional coordinate system20 so that a sensor-to-camera distance Dsc remains constant.Additionally, a relative orientation between the camera 14 and thedistance sensor 16 remains constant within the coordinate system 20 asthe camera 14 and the distance sensor 16 move around the object 12 tocapture multiple images of the object 12 from different angles. Tomaintain the fixed spatial relationship, the camera 14 and the distancesensor 16 may be mounted to a common structure (not shown) such as apan-tilt unit, an articulating arm, a tripod or other support structurethat can move or be moved around the object 12 in two or threedimensions.

In the position of the camera vision system 10 shown in FIG. 1, thecamera 14 is positioned at a first camera-to-object distance Dco fromthe object 12. At the same time, the distance sensor 16 is positioned ata sensor-to-object distance Dso. With the camera 14 and the distancesensor 16 having the fixed spatial relationship within the coordinatesystem 20, the camera-to-object distance Dco can be calculated based onthe known sensor-to-object distance Dso using standard geometriccalculations. In some embodiments, the conversion from thesensor-to-object distance Dso to the corresponding camera-to-objectdistance Dco and camera focus state may be pre-calculated and stored ina lookup table at the system controller 18, and are retrieved when thesensor-to-object distance Dso is detected. Once determined, the systemcontroller 18 transmits control signals to the camera 14 to shift itsfocus to the new focus state. As long as the camera 14 and the distancesensor 16 remain at the same distances Dco, Dso, respectively, from theobject 12, the camera 14 maintains its focus state. In FIG. 2, thecamera 14 and the distance sensor 16 have articulated to a new positionrelative to the object 12. The camera 14 and the distance sensor 16 havetranslated and rotated within the coordinate system 20, but thesensor-to-camera distance Dsc remains constant due to their fixedspatial relationship. Distance sensor signals from the distance sensor16 indicate to the system controller 18 that the distance sensor 16 isat a second sensor-to-object distance Dso2 from the object 12. Upondetecting the change, the system controller 18 determines acorresponding second camera-to-object distance Dco2 and a second focusstate for the camera 14. The system controller 18 then transmits camerafocus control signals to the camera 14 to shift its focus to the secondfocus state.

In some embodiments, the camera vision system 10 may include multiplecameras that move together to capture multiple images of the object 12for use in the 3D reconstruction the object 12. Referring to FIG. 3, thecamera vision system 10 includes a second camera 22 having a fixedspatial relationship with the first camera 14 and the distance sensor16. The second camera 22 is a second sensor-to-camera distance Dsc2 fromthe distance sensor 16, and the system controller 18 is configured todetermine a second camera-to-object distance Dc2o from the second camera22 along with the first camera-to-object distance Dc1o of the firstcamera 14. The system controller 18 also determines a focus state forthe second camera 22 and transmits camera focus control signals to thesecond camera 22 to cause the second camera 22 to adjust to the focusstate for capturing images of the object 12.

To convert the images of the object 12 into a reconstructed 3-D image,certain intrinsic parameters of the cameras 14, 22 are required for anaccurate conversion. Such intrinsic properties can include the focallength of the camera, the location of the principal point on thecamera's focal plane, tangential and radial distortion coefficients andthe like. Values for the intrinsic parameters can be calculated for eachcamera or model of camera using known techniques for intrinsiccalibration. An example of intrinsic calibrations and 3-D reconstructiontechniques can be found athttps://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html.Such techniques may include focusing on and capturing images of anobject having a known geometry, such as a chessboard, and determining ifand how much the captured image is distorted relative to the geometry ofthe reference object.

For a given camera, each focus or focus state of the camera has a uniqueset of values of the intrinsic parameters. Therefore, to achieve anaccurate and well-defined 3D reconstruction of an object, intrinsicparameter values for a range of focus states may be determined andstored for later use. The range of focus states may extend from amaximum focus state to a minimum focus state, and include focus statesat regular intervals between the minimum and maximum focus states. Ateach focus state within the range, the camera may be set to thecorresponding focus, and intrinsic parameter values may be calculatedusing traditional intrinsic calibration techniques such as thosedescribed above. The focus states and corresponding intrinsic parametervalues may be used to populate an intrinsic parameter database for thecamera. With the intrinsic parameter database populated, the deviationbetween the intrinsic parameter values for successive focus states maybe assessed. If the difference is too great to construct sufficientlywell-defined 3D images, the focus state interval may be reduced andintrinsic parameter values may be determined for additional focus statesuntil the intrinsic parameter value deviations are within tolerance toconstruct well-defined 3D images.

In the camera vision system 10 in accordance with the presentdisclosure, the distance from the distance sensor 16 and the determinedintrinsic parameter values for the focus states can be used in a dynamicintrinsic process to create well-defined 3D reconstructions of theobject 12. The distance sensor 16 provides the sensor-to-object distanceDso to the system controller 18 that in turn determines the appropriatefocus state. The system controller 18 transmits control signals to thecamera 14 to cause the camera 14 to adjust the focus state. The systemcontroller 18 also retrieves the corresponding intrinsic parametervalues for the focus state from the intrinsic parameter database. If thecalculated focus state is between focus states stored in the intrinsicparameter database, the system controller 18 may interpolate the storedintrinsic parameter values to generate intrinsic parameter values forthe calculated focus state. Captured image data from the camera 14 iscombined with the retrieve and/or interpolated intrinsic parametervalues for use in the 3D reconstruction of the object 12. Embodiments ofthe dynamic intrinsic process are illustrated and described in greaterdetail below.

FIG. 4 illustrates an embodiment of electrical and electro-mechanicalcontrol elements of the camera vision system 10. The system controller18 may be capable of processing information received from the cameras14, 22, the distance sensor 16 and other monitoring and control devicesusing software stored at the system controller 18, and outputtingcommand and control signals to the cameras 14, 22 and other devices ofthe camera vision system 10. The system controller 18 may include aprocessor 30 for executing a specified program, which controls andmonitors various functions associated with the camera vision system 10.The processor 30 may be operatively connected to a memory 32 that mayhave a read only memory (ROM) 34 for storing programs, and a randomaccess memory (RAM) 36 serving as a working memory area for use inexecuting a program stored in the ROM 34. Although the processor 30 isshown, it is also possible and contemplated to use other electroniccomponents such as a microcontroller, an application specific integratedcircuit (ASIC) chip, or any other integrated circuit device.

The cameras 14, 22 are operatively connected to the system controller18. The cameras 14, 22 may be any appropriate camera configured forcapturing high-resolution images and having lens and focus control thatcan be integrated into the camera vision system 10 in accordance withthe present disclosure. As illustrated, each camera 14, 22 hasintelligence provided by a camera controller 40 that may be of a typesuch as those discussed above for the system controller 18. The cameracontroller 40 is configured to exchange control signals and data withthe system controller 18. For example, the system controller 18 maytransmit focus and image capture control signals, and the cameracontroller 40 may transmit image data for captured images of the objects12. The camera controller 40 may be programmed to automatically focusthe camera 14, 22 in the absence of focus control signals from thesystem controller 18 using auto-focusing techniques known in the art.Software for such techniques may be stored in a camera memory (notshown) and accessed by the camera controller 40.

The illustrated cameras 14, 22 further include camera lens actuators 42operatively coupled to the camera controller 40. The camera lensactuator 42 may be operatively connected to a lens (not shown) of thecamera 14, 22 and actuatable to move the lens and adjust the focus stateof the camera 14, 22. The camera lens actuator 42 and the connection tothe lens may be any appropriate electro-mechanical mechanism configuredto convert movement of the camera lens actuator 42 into movement of thecamera lens. In one embodiment, the camera lens actuator 42 comprises arotary encoder wherein each discrete encoder position corresponds to afocus state of the camera 14, 22. When a focus state is determined bythe system controller 18, the focus state may be converted to thecorresponding encoder position value, and the camera lens actuator 42 iscontrolled to actuate and rotate to the encoder position. The use ofcameras 14, 22 having other types of camera lens actuators 42 in thecamera vision system 10 is contemplated by the inventors.

The distance sensor 16 of the camera vision system 10 is operativelyconnected to the system controller 18 to provide distance sensor signalsindicative of the sensor-to-object distance Dso to the sensed object 12.The distance sensor 16 may be any appropriate sensor capable ofdetecting an object and sensing a distance to the object. Exemplarydistance sensor 16 may include point-source lasers, optical sensors,radar sensors, sonic sensors, ultrasonic sensors and the like. Theparticular distance sensor 16 implemented in the camera vision system 10may have a resolution necessary to accurately determine a focus statefor the cameras 14, 22 and retrieve/interpolate the correspondingintrinsic parameter values.

As part of or in addition to the memory 32 of the system controller 18,databases may be used to store information necessary for image captureand 3D reconstruction in the camera vision system 10. An intrinsicparameter database 50 may be provided to store the focus state andcorresponding intrinsic parameter values determined during the intrinsiccalibration of the cameras 14, 22. The intrinsic parameter database 50may be populated during the intrinsic calibration, and later accessedduring image capture and/or 3D reconstruction. The camera vision system10 may further include an image database 52 that stores two-dimensional(2D) image data captured by the cameras 14, 22 as images of the object12 are captured. Along with the image data, the image database 52 maystore the corresponding focus state at which each captured image wastaken. The focus state can then be used to retrieve/interpolate thecorresponding intrinsic parameter values from the intrinsic parameterdatabase 50. In alternative embodiments, the intrinsic parameter valuesmay also be stored with the image data if doing so facilitates the 3Dreconstruction of the object 12 and reduces duplicate processing steps.

FIG. 4 illustrates one embodiment of the components of the camera visionsystem 10, but alternative embodiments are contemplated by theinventors. For example, the processing of the controllers 18, 40 can becombined at the system controller 18, with the system controller 18controlling the camera lens actuators 42 directly. In other embodiments,the distance sensor 16 may be operatively connected to each cameracontroller 40, and with the camera controls 40 being programmed toconvert the sensor-to-object distance Dso to the correspondingcamera-to-object distances Dco. In other embodiments, intrinsicparameter values may be stored for each individual camera 14, 22 of thecamera vision system 10. Alternatively, the camera vision system 10 maystore a single set of intrinsic parameter values for each model ofcamera used in the camera vision system 10 so long as there are notsignificant variations in the intrinsic calibrations of the same modelcameras. The intrinsic parameter database 50 and the image database 52may be stored at the same memory structure, such as the memory 32 of thesystem controller 18, or may be stored in different memory structuresthat are each connected to and accessed by the system controller 18during intrinsic calibration of the cameras 14, 22 and image capture and3D reconstruction of the object 12. Further alternative configurationsof the components of the camera vision system 10 will be apparent tothose skilled in the art and are contemplated by the inventors as havinguse in camera vision systems 10 and image capture and 3D reconstructionroutines in accordance with the present disclosure.

INDUSTRIAL APPLICABILITY

FIG. 5 illustrates a flow diagram of an exemplary camera vision systemintrinsic calibration routine 100 in accordance with the presentdisclosure. The intrinsic calibration routine 100 may begin at a block102 where the camera 14, 22 to be calibrated is adjusted to an initialfocus state of a focus state range of values. The focus state range ofvalues may reflect a range of distances at which the cameras 14, 22 maybe positioned relative to the object 12. For example, the cameras 14, 22may have potential distances to the object 12 ranging from a minimum oftwo feet to a maximum of thirty feet from the object 12. To start theintrinsic calibration routine 100, the camera may be adjusted to focustwo feet from the camera 14, 22. In subsequent iterations as discussedfurther below, the focus state may be adjusted by specified focus stateincrements up to the maximum focus state.

After the camera 14, 22 is set to the initial focus state at the block102, control may pass to a block 104 where the camera 14, 22 captures animage of a target object. The target object may have a known geometry,such as a chessboard so that distortions in the captured image from theknown geometry of the target object may be identified and quantified.The image capture by the camera 14, 22 may be controlled by the systemcontroller 18, the corresponding camera controller 40, or othercontroller that is managing the process of the intrinsic calibrationroutine 100. After the image is captured at the block 104, control maypass to a block 106 where the system controller 18, the cameracontroller 40 or other controller determines the values of the intrinsicparameters for the current focus state of the camera 14, 22. Intrinsicparameter values may be determined using any known techniques forintrinsic calibration of a camera, such as those described above. Afterdetermining the intrinsic parameter values at the block 106, control maypass to a block 108 where the current focus state and the intrinsicparameter values for the current focus state are stored in a storagestructure such as the intrinsic parameter database 50.

The intrinsic calibration routine 100 continues by acquiring intrinsicparameter values at a plurality of intermediate focus states between theminimum focus state and the maximum focus state. The intermediate focusstates may occur at predetermined focus state increments until the lastor maximum focus state is calibrated. In the present example, the focusstates of the camera 14, 22 may be increased at six inch increments fromthe two foot minimum focus state up to the thirty foot maximum focusstate. Consequently, after the intrinsic parameter values are stored atthe block 108 for the two foot focus state, control may pass to a block110 where the controller executing the intrinsic calibration routine 100may determine if the last focus state of the focus state range has beencalibrated. If the last focus state has not been calibrated, such asafter only the minimum focus state has been calibrated, control may passto a block 112 where the focus state of the camera 14, 22 is adjusted bythe focus state increment. In the example, the camera 14, 22 is adjustedfrom the two foot focus state to the 2.5 foot state. After the camera14, 22 is adjusted to the new focus state, control passes back to theblocks 104-108 to capture an image of the target object, and determineand store the intrinsic parameter values for the new focus state.

If it is determined that the last focus state has been calibrated at theblock 110, such as when the thirty foot focus state has been calibrated,control may pass to a block 114 to determine the deviations between theintrinsic parameter values for adjacent focus states. In a simpleimplementation, the deviations between the intrinsic parameter values ofthe focus states may be determined by subtraction to find the differencebetween the intrinsic parameter values. In other implementations, morecomplex mathematical and statistical methods may be implemented todetermine deviations between the intrinsic parameter values. Deviationsin the intrinsic parameter values that are too large or greater than aspecified tolerance may result in 3D reconstructions having poor qualityand poor definition, and may require calibration of additionalintermediate focus states to improve the quality of the 3Dreconstruction of the object 12.

After the deviations in the intrinsic parameter values are determined atthe block 114, control passes to a block 116 where the controllerdetermines whether the deviations are too large to produce qualityimages. If the deviations are not too large, the intrinsic calibrationof the camera 14, 22 may be sufficient to produce high-quality 3Dreconstructions of the objects 12. In this case, the intrinsic parameterdatabase 50 may be sufficiently populated and the intrinsic calibrationroutine 100 may terminate.

If the deviations are too large, additional focus states at smallerincrements between the ends of the focus state range may be taken toreduce the deviations between the intrinsic parameter values of adjacentfocus states. When the deviations are too large, control may pass to ablock 118 where the focus state increment may be reduced by anappropriate amount so that focus states between previously calibratedfocus states will be calibrated and stored in the intrinsic parameterdatabase 50. For example, the six-inch focus state increment in thepresent example may be reduced to a three-inch increment to double thedensity of focus states that are calibrated by the intrinsic calibrationroutine 100. After the focus state increment is adjusted at the block118, control may pass back to the block 102 to readjust the camera 14,22 to the initial focus state and cycle through the focus state range tocalibrate the additional focus states. In some embodiments, theintrinsic calibration routine 100 may be configured to skip focus statesthat have already been calibrated in previous iterations of theintrinsic calibration routine 100 in order to reduce the time andprocessing resources required to fully calibrate the camera 14, 22. Theiterative process of the intrinsic calibration routine 100 will continueuntil the deviation in the intrinsic parameter values is withinacceptable tolerances at the block 116. The intrinsic calibrationroutine 100 may be repeated for each of the cameras used in the cameravision system 10. If the precision of manufacturing the cameras 14, 22of a particular camera model is high and the intrinsic parameter valueswill not vary substantially from camera 14, 22 to camera 14, 22, it maybe sufficient to calibrate one exemplary camera 14, 22 of that model andstore the calibrated intrinsic parameter values for use with the imagedata captured by a camera 14, 22 of that model in the 3D reconstructionprocess.

Once the cameras 14, 22 of the camera vision system 10 have beencalibrated, the camera vision system 10 may be used to capture images ofthe object 12 that may then be converted into a 3D reconstruction of theobject 12. FIG. 6 illustrates an embodiment of a flow diagram of anexemplary image capture and 3D reconstruction routine 150 in accordancewith the present disclosure. The routine 150 may begin at a block 152where the camera or cameras 14, 22 of the camera vision system 10 arepointed at the object 12 to be imaged. As discussed above, the cameras14, 22 may be mounted to a common structure such as a pan-tilt unit thatcan move or be moved around the object 12 in two or three dimensions.The structure may be manually movable by an operator or technician toposition the cameras 14, 22 at various locations necessary to captureimages for an accurate 3D representation of the object 12.Alternatively, movement of the structure and the cameras 14, 22 mountedthereon may be automated and controlled by a control structure such asthe system controller 18 to move the cameras 14, 22 to various imagecapture locations about the object 12.

Once the cameras 14, 22 are positioned and pointed at the object 12,control may pass to a block 154 where the distance sensor 16 senses thesensor-to-object distance Dso at the current position. The distancesensor 16 senses the sensor-to-object distance Dso and transmitsdistance sensor signals having values corresponding to thesensor-to-object distance Dso to the system controller 18. In responseto receiving the distance sensor signals from the distance sensor 16 atthe system controller 18, control may pass to a block 156 where thesystem controller 18 determines the camera-to-object distances Dco basedon the sensor-to-object distance Dso in the distance sensor signals. Thecamera-to-object distances Dco may be calculated in any appropriatemanner as described above.

If the camera-to-object distance Dco has not changed since lastdetermined at the block 156, the focus state of the cameras 14, 22 hasnot changed and it may not be necessary to refocus the cameras 14, 22and retrieve intrinsic parameter values for the focus state.Consequently, if the camera-to-object distance Dco has not changed atthe block 158, control of the routine 150 may bypass camera focusing andintrinsic parameter value retrieval steps, and instead proceed tocapturing an image of the object 12. However, if the camera-to-objectdistance Dco has changed at the block 158, control may pass to a block160 where the system controller 18 may determine the focus state for thecameras 14, 22 corresponding to the sensor-to-object distance Dso andthe camera-to-object distances Dco, and to a block 162 to cause thecameras 14, 22 to be adjusted to the new focus state. In one embodiment,the system controller 18 may transmit camera control signals to thecamera controller 40 containing values corresponding to the focus state.The camera controller 40 may receive the camera control signals, andconvert the focus state information to camera lens actuator controlsignals that are transmitted to the camera lens actuator 42. The cameralens actuator control signals will cause the camera lens actuator 42 toactuate and move the lens of the camera 14, 22 to the positioncorresponding to the new focus state. In alternative embodiments, thesystem controller 18 may format and transmit the camera lens actuatorcontrol signals to the camera lens actuator 42 directly.

After the camera focus state is adjusted at the block 162, control maypass to a block 164 where the system controller 18 may retrieveintrinsic parameter values corresponding to the camera focus state fromthe intrinsic parameter database 50. The intrinsic parameter values willbe needed during the 3D reconstruction process to accurately convertimages of the object 12 taken by the camera 14, 22 at the current camerafocus state. If the camera focus state matches a stored camera focusstate, the corresponding intrinsic parameter values may be used in thesubsequent steps of the routine 150. In other instances, the camerafocus state determined from the distances Dso, Dco may fall betweencamera focus states stored in the intrinsic parameter database 50. Insuch cases, the system controller 18 may be programmed to retrieve andinterpolate the intrinsic parameter values for relevant stored camerafocus states to generate intrinsic parameter values that more closelycorrespond to the current camera focus state of the camera 14, 22.

With the camera 14, 22 adjusted to the camera focus state and thecorresponding intrinsic parameter values retrieved and/or interpolatedfrom the intrinsic parameter database 50, either during the presentiteration or during a previous iteration of the routine 150, control maypass to a block 166 where an image of the object 12 may be captured bythe cameras 14, 22 at the current camera focus state. Depending on theconfiguration of the camera vision system 10, the system controller 18may transmit image capture control signals to the camera controller 40to cause the camera 14, 22 to snap an image of the object 12. After theimage of the object 12 is captured by the camera 14, 22, the cameracontroller 40 may transmit object image data for the captured image backto the system controller 18. When the object image data is received atthe system controller 18, control may pass to a block 168 where theobject image data and the intrinsic parameter values for the camerafocus state are stored in the image database 52 for later use during the3D reconstruction of the object 12.

After the object image data and the intrinsic parameter values arestored in the image database 52, the system controller 18 may determinewhether the routine 150 is to continue capturing images of the object12. In an automated image capture process, the system controller 18 maybe programmed with a sequence of locations in which to position thecamera 14, 22 to capture a sufficient number of images of the object 12to generate a high-quality 3D reconstruction of the object 12. If thesystem controller 18 determines that images of the object 12 have notbeen captured at each of the locations in the sequence at a block 170,control may pass back to the block 152 to reposition the camera 14, 28at the next location and point the camera at the object 12. In a manualor partially manual implementation of the image capture process, anoperator or technician may enter coordinates corresponding to the nextlocation at which the camera 14, 22 will capture an image of the object12, thereby causing control the pass back to the block 152. Otherimplementations of the routine 150 are contemplated where thedetermination step at the block 170 may occur automatically,semi-automatically or manually so that the routine 150 will continueuntil the camera 14, 22 has been positioned at all the requiredlocations for the 3D reconstruction process and has captured images ofthe object 12 at those locations. Once it is determined that no moreimages of the object 12 will be captured at the block 170, control maypass to a block 172 where the 3D reconstruction of the object 12 isperformed utilizing any appropriate reconstruction technique asdiscussed above utilizing the object image data and correspondingintrinsic parameter values stored in the image database 52.

Traditional camera intrinsic calibration methods and image capture donot incorporate dynamic focusing of camera lenses as distances betweencameras and target objects change. Camera vision systems 10 inaccordance with the present disclosure provide and external distancesensor 16 that provides object distance information that is used tocontrol the adjustment of cameras 14, 22 to appropriate focus states forcapturing images of the target object 12, and to retrieve the applicableintrinsic parameter values for focus state of the camera 14, 22 so thataccurate, high quality 3D reconstructions of the target object 12 can begenerated. Adjusting the camera focus state based on thesensor-to-object distance Dso provided by the distance sensor 16 mayyield a fast, efficient and accurate auto-focusing response in thecameras 14, 22. Moreover, converting from the sensor-to-object distanceDso to the corresponding camera focus state and retrieving thecorresponding stored intrinsic parameter values for the camera focusstate can improve the speed and accuracy of the process forreconstructing 3D images of the target object 12, and improve thequality of the images. In implementations where the quality of the 3Dreconstruction may be crucial, such as where the 3D representations ofthe object 12 are part of an inspection process, the inspection time maybe reduce by increasing the accuracy and quality of the 3D reconstructedimage reviewed by the inspectors.

While the preceding text sets forth a detailed description of numerousdifferent embodiments, it should be understood that the legal scope ofprotection is defined by the words of the claims set forth at the end ofthis patent. The detailed description is to be construed as exemplaryonly and does not describe every possible embodiment since describingevery possible embodiment would be impractical, if not impossible.Numerous alternative embodiments could be implemented, using eithercurrent technology or technology developed after the filing date of thispatent, which would still fall within the scope of the claims definingthe scope of protection.

It should also be understood that, unless a term was expressly definedherein, there is no intent to limit the meaning of that term, eitherexpressly or by implication, beyond its plain or ordinary meaning, andsuch term should not be interpreted to be limited in scope based on anystatement made in any section of this patent (other than the language ofthe claims). To the extent that any term recited in the claims at theend of this patent is referred to herein in a manner consistent with asingle meaning, that is done for sake of clarity only so as to notconfuse the reader, and it is not intended that such claim term belimited, by implication or otherwise, to that single meaning.

What is claimed is:
 1. A camera vision system for creating athree-dimensional (3D) reconstruction of an object comprising: a camerahaving a camera controller; a distance sensor having a fixed spatialrelationship with the camera; and a system controller operativelyconnected to the camera controller and the distance sensor, the systemcontroller being programmed to: receive distance sensor signals from thedistance sensor, wherein the distance sensor signals indicate asensor-to-object distance from the distance sensor to the object,determine a camera-to-object distance from the camera to the object anda corresponding camera focus state for the camera-to-object distancebased on the sensor-to-object distance, transmit camera focus statecontrol signals to the camera controller to cause the camera controllerto adjust the camera to the camera focus state, retrieve cameraintrinsic parameter values for the camera that correspond to the camerafocus state, transmit image capture control signals to the cameracontroller to cause the camera to capture an object image of the object,receive object image data from the camera controller corresponding tothe object image captured by the camera, and store the object image dataand the camera intrinsic parameter values in an image database.
 2. Thecamera vision system of claim 1, wherein the camera comprises a cameralens actuator operatively connected to the camera controller, whereinthe system controller is programmed to convert the camera-to-objectdistance to a camera lens actuator position corresponding to the camerafocus state, and wherein the camera focus state control signals includethe camera lens actuator position.
 3. The camera vision system of claim1, wherein the system controller is programmed to: detect a change ofthe sensor-to-object distance to a second sensor-to-object distance fromthe distance sensor to the object based on the distance sensor signals;and determine a second camera-to-object distance from the camera to theobject and a corresponding second camera-to-object focus state for thesecond camera-to-object distance based on the second sensor-to-objectdistance.
 4. The camera vision system of claim 1, comprising anintrinsic parameter database storing a plurality of camera focus statesand corresponding camera intrinsic parameter values for each of theplurality of camera focus states, wherein the system controller isoperatively connected to the intrinsic parameter database and isprogrammed to retrieve the camera intrinsic parameter values for thecamera from the intrinsic parameter database.
 5. The camera visionsystem of claim 1, wherein the system controller is programmed toperform a 3D reconstruction of the object from stored object image dataand corresponding camera intrinsic parameter values.
 6. The cameravision system of claim 1, comprising a second camera having a secondcamera controller, wherein the second camera has a second fixed spatialrelationship with the distance sensor, wherein the system controller isoperatively connected to the second camera controller, and wherein thesystem controller is programmed to: determine a second camera-to-objectdistance from the second camera to the object and a corresponding secondcamera focus state for the second camera-to-object distance based on thesensor-to-object distance, transmit second camera focus state controlsignals to the second camera controller to cause the second cameracontroller to adjust the second camera to the second camera focus state,retrieve second camera intrinsic parameter values for the second camerathat correspond to the second camera focus state, transmit image capturecontrol signals to the second camera controller to cause the secondcamera to capture a second object image of the object, receive secondobject image data from the second camera controller corresponding to thesecond object image captured by the second camera, and store the secondobject image data and the second camera intrinsic parameter values inthe image database.
 7. The camera vision system of claim 6, wherein thesystem controller is programmed to perform a 3D reconstruction of theobject from stored object image data and corresponding first cameraintrinsic parameter values, and stored second object image data andcorresponding second camera intrinsic parameter values.
 8. A method ofdynamically focusing a camera and capturing images and generatingthree-dimensional (3D) reconstructions of objects, comprising:determining a first camera-to-object distance from the camera to anobject; causing a camera controller of the camera to adjust a camerafocus state of the camera to a first camera focus state that correspondsto the first camera-to-object distance; retrieving first cameraintrinsic parameter values of the camera that correspond to the firstcamera focus state; commanding the camera controller to cause the camerato capture a first image of the object; and storing first image data ofthe first image and the first camera intrinsic parameter values.
 9. Themethod of claim 8, wherein determining the first camera-to-objectdistance comprises: determining a first sensor-to-object distance from adistance sensor to the object, wherein the distance sensor has a fixedspatial relationship with the camera; and determining the firstcamera-to-object distance from the camera to the object based on thefirst sensor-to-object distance.
 10. The method of claim 8, whereincausing the camera controller to adjust the camera focus statecomprises: retrieving a first camera lens actuator setting for a cameralens actuator of the camera, wherein the first camera lens actuatorsetting corresponds to the first camera focus state; and causing thecamera lens actuator to displace to the first camera lens actuatorsetting to adjust the camera to the first camera focus state.
 11. Themethod of claim 8, wherein retrieving first camera intrinsic parametervalues comprises retrieving the first camera intrinsic parameter valuesfrom an intrinsic parameter database that stores a plurality of camerafocus states and corresponding intrinsic parameter values of the camerafor each of the plurality of camera focus states.
 12. The method ofclaim 8, wherein storing the first image data comprises storing thefirst image data and the first camera intrinsic parameter values in animage database with additional images of the object and correspondingcamera intrinsic parameter values taken from multiple camera positionsrelative to the object.
 13. The method of claim 8, comprising performingthe determining, causing, retrieving, commanding and storing steps ateach of a plurality of camera positions relative to the object.
 14. Themethod of claim 13, comprising performing a 3D reconstruction of theobject from stored image data and corresponding camera intrinsicparameter values.
 15. A method for performing intrinsic calibration in acamera vision system, the method comprising: adjusting a camera of thecamera vision system to a camera focus state; capturing an image of atarget object; determining intrinsic parameter values for the camera atthe camera focus state from captured target image data for the image ofthe target object; and storing the camera focus state and the intrinsicparameter values for the camera at the camera focus state in anintrinsic parameter database.
 16. The method of claim 15, comprising;determining intrinsic parameter data for the camera at a range of camerafocus states from a minimum camera focus state to a maximum camera focusstate; and storing the range of camera focus states and correspondingintrinsic parameter values for the camera at the range of camera focusstates in the intrinsic parameter database.
 17. The method of claim 16,wherein camera focus states in the range of camera focus statesincreases by a predetermined focus state increment from the minimumcamera focus state to the maximum camera focus state.
 18. The method ofclaim 17, comprising: determining an intrinsic parameter deviationbetween the intrinsic parameter values of the camera focus states in therange of camera focus states; and determining intrinsic parameter valuesfor the camera for additional camera focus states between the minimumcamera focus state and the maximum camera focus state in response todetermining that the intrinsic parameter deviation is greater than apredetermined intrinsic parameter deviation tolerance.
 19. The method ofclaim 15, wherein the target object has a known geometry.
 20. The methodof claim 15, comprising determining and storing intrinsic parametervalues at camera focus states for a plurality of cameras of the cameravision system.